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The purpose of this annotated bibliography was to inform an effort to create 
instruments that would yield indicators of the extent to which the Common Core State 
Standards for Mathematics or similar standards are implemented in classroom instruction. 
Reviewed here are instruments reflecting: coverage of classroom content, use of specific 
instructional practices, and use of textbooks in K-12 mathematics classrooms in the United 
States. 

To identify articles within the parameters, we used the search criteria 
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“mathematics” paired with “instructional practice,” “content,” “curriculum,” or “textbook.” 
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Additional searches paired the terms “reform,” “implementation, 
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application,” 
“measuring,” and “common core,” as well as different variants, with “mathematics”. This 
extensive search, which was limited to articles in English, yielded 196 articles. We also 
consulted the references found in these articles. 

Of the 196 articles identified in the extensive search, 63 were considered relevant for 
our interest in classroom content coverage. These relevant articles focused on analyzing and 
capturing the implementation of mathematics curriculum content, instructional practices, and 
textbook use. Within this collection of articles, this review utilizes 20 papers that developed 
instruments to capture one or more of these three aspects of instruction. 

The articles reviewed included use of a variety of statistical instruments including 
cross-national surveys implemented by international organizations (e.g., PISA, TIMSS and 
TALIS), national evaluations (e.g., the National Council of Teachers of Mathematics 
(NCTM) and the National Assessment of Educational Progress (NAEP)), and state-level 


evaluations in the United States. Although many papers based on these large studies are 


included in the references, a summary of each one was deemed beyond the scope of this 
project. 

The relevant literature covers a wide variety of education indicators. There were, 
however, a number of performance measures which were common among the literature. 
Recurrent performance measures include analysis of the depth of specific curricular reforms, 
time spent on each practice, instrument validity and reliability, teacher interpretation of 
standards, appraisal of teacher beliefs, textbook alignment with standards, and teacher 
preparation. 

While we identified a wide variety of instruments implemented throughout the 
relevant articles, a small number were used most frequently. These instruments included 
surveys (of teachers, principals, and students), in-depth interviews (of teachers, textbook 
authors, and math specialists), classroom observations, teacher logs, vignette-based 
measures, and textbook examinations. 

Comprehensive Assessments of Classroom Content, Instructional Practices, and Use of 
Textbooks 

The literature review revealed that teacher surveys are the most commonly applied 
method to capture mathematics classroom content, instructional practices, and use of 
textbooks. We also found that almost half of the studies reviewed have simultaneous 
assessment of classroom content, instructional practices, and/or use of textbooks. 

Among the most comprehensive work we found is a series of papers and reports 
evaluating the implementation of curricular reform. One of the most thorough reports on 


teacher content, practices, and use of textbooks is Porter, Kirst, Osthoff, Smithson, and 


Schneider’s (1993) report “Reform up Close: An Analysis of High School Mathematics and 
Science Classrooms.” The study assessed curriculum implementation within six states, 12 
districts, and 18 schools. Instruments applied included teacher questionnaires; and 
interviews with principals, district administrators and agency administrators. These 
instruments collected detailed information regarding curriculum implementation at the 
classroom level. For example, one survey question inquired about time and depth of 
coverage devoted to 80 specific mathematics topics. 

Another influential study, “Validating National Curriculum Indicators,” was part of 
the Rand Corporation project (Burstein et al., 1995). This study aimed to improve the quality 
of information collected on school curriculum. The teacher survey that was implemented 
collected detailed information regarding math content, teaching practices, and textbook use. 
The survey was complemented with teacher logs and an evaluation of assignments and 
exams. 

The “Study of Instructional Improvement” conducted by the University of Michigan 
and Consortium for Policy Research of Education was a large scale longitudinal study of 
instructional improvement. As part of this project several journal articles and working papers 
have been published, including Rowan, Harrison, and Hayes, (2004) and Hill (2005). Ball, 
Cohen, and Rowan (2010) describes the mathematics curriculum and teaching practices in a 
sample of high-poverty elementary schools that participated in three of the most widely 
implemented school reform programs in the United States: Accelerated Schools Program, 
America’s Choice, and Success for All. The study evaluated the content coverage and 


teaching practices within and among 53 schools using instructional logs. 


Also stemming from the “Study of Instructional Improvement”, Hill (2005) 
examined a set of terms often found on survey instruments describing mathematical content. 
To do so, she used a field pilot of an elementary mathematics daily log to examine terms 
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such as “geometry,” “number patterns,” and “ordering fractions” for rates of agreement and 
disagreement between teachers and observers. . The study assessed when and how 
disagreements occurred through the use of interviews and written observations. 

Swanson and Stevenson (2002) presented results of a teacher survey focused on 
classroom content and practices in states that participated in the NAEP. The study examined 
if there was an association between the structure of the national standards movement and 
patterns of state-level policy activism. 

Porter, Smithson, Blank, & Zeidner (2007) developed procedures to investigate the 
agreement between content standards, tests, textbooks, and even classroom instruction as 
experienced by teachers and students. The purpose was to define and illustrate innovative 
uses of tools, content maps, and a quantitative index of the degree of agreement. This 
research drew on data from Porter, Blank, Smithson, and Osthoff (2005) that implemented a 
place-based randomized trial to test the effects on instructional practices of mathematics and 
science in a professional development program in 50 U.S. middle schools within five large 
urban districts. They concluded that “‘a quantitative measure of alignment can be defined at 
the individual teacher level and used as an important variable in research on teaching” (Porter 
et al., 2007, p. 46). 

Gimbert, Bol, and Wallace (2007) analyzed the influence of traditional and 


alternative teacher preparation programs on student achievement and instruction in urban 


secondary schools. They utilized a mixed-method comparative design to determine if teacher 
preparation influences the implementation of the process standards established by the 
NCTM. This study used classroom observations to determine the extent to which the teachers 
implemented NCTM process and content standards. 

Blank, Porter, and Smithson (2001) was part of a project to develop and test Surveys 
of Enacted Curriculum in mathematics and science. The survey was designed to address 
broad questions about standards-based reform by testing a survey approach to analyzing the 
enacted curriculum in mathematics and science. More than 600 teachers across the 11 
participating states completed self-report surveys about the subject content and the 
instructional practices used in their classes. 

Gagnon, and Maccini (2007) assessed teachers’ definition of math, familiarity with 
the course topics, effectiveness of methods courses, preparation to use and frequency of use 
of effective instructional strategies, and factors contributing to the use of instructional 
strategies. They used mail-based surveys of special and general education high school 
teachers that teach math to students with emotional and behavior disorders and learning 
disabilities. This paper emphasized the need for better preparation for instructional strategies 
that are consistent with the goals of the NTCM standards and empirically-validated teaching 
practices (Gagnon & Maccini, 2007). 

McCaffrey et al. (2001) examined how student achievement is related to the degree 
to which teachers' use of instructional practices align with reforms. The study drew on 
findings from teacher surveys and components of the Stanford achievement test for tenth 


grade students. The analysis controlled for student background characteristics and past 


achievement. Particular focus was placed on how curriculum reforms might influence the 
relationship between instructional practices and student outcomes. Results from all sites were 
combined to produce a single coefficient estimate for instructional practices. 

Lane and Parke (2008) examined the extent to which mathematics classroom 
activities in Maryland were aligned with state learning outcomes and the Maryland School 
Performance Assessment Program. The authors collected information on activities related to 
instruction, assessment, and test-preparation from a statewide stratified random sample of 
teachers. Seven types of teachers’ classroom activities were analyzed: (1) mathematics 
process outcomes; (2) mathematics content outcomes; (3) student response types; (4) 
interpretation of charts, tables, and graphs; (5) use of manipulatives and calculators; (6) 
integration with other subject areas; and (7) overall agreement with the Maryland State 
Performance Assessment Program. Most classroom activities were aligned with aspects of 
state assessment and standards. 

Stecher et al. (2006) used vignette-based measures of reform-oriented instruction, 
which is one aspect of mathematics instructional practice. Teachers read contextualized 
descriptions of teaching practices that varied in terms of reform-oriented instruction, and then 
rated the degree to which the options correspond to their own likely behaviors. 

Ross, McDougall, Hogaboam-Gray, and LeSage (2003) evaluated instruments that 
measure teachers' instructional practice. To examine the extent to which teachers implement 
mathematics education reforms, they developed a twenty-item survey based on nine 
dimensions of standards-based teaching. They provide evidence of the reliability and validity 


of the instrument using correlations of survey scores with a mandated performance 


assessment in Grade 6 mathematics and classroom observations of a small sample of 
teachers. The study also demonstrated that teachers who were similar in their claims about 
using a standards-based text series differed in how they used the text in the classroom in 
ways predicted by the survey. Similarly, Mayer (1999) examined the reliability and validity 
of using survey instruments to capture instructional practices. The study inspected if the 
portion of time devoted to established practices in algebra classes were consistent with 
NCTM professional standards. 

Spillane and Zeuli (1999) investigated teacher math practices in the context of the 
current reform effort in the United States. The study developed a conceptual framework for 
examining reform proposals “in terms of the intersection of classroom tasks and discourse 
patterns with principled and procedural mathematical knowledge” (p. 1). The student 
identified which teacher practices were most responsive to reform through use of classroom 
observation and teacher interviews. 

Hamilton et al. (2003) analyzed the relationship between academic achievement and 
classroom practices established by the National Science Foundation's Systemic Initiatives 
(SI) program. The study collected data from eleven SI sites and investigated relationships at 
the site level and pooled across sites using a planned meta-analytic approach. The study 
found a small but consistent positive relationship between teachers' reported use of 
standards-based instruction and student achievement. 

Jitendra et al. (2005) examined the extent to which teaching the recommended 
methods for problem-solving recommended in mathematic textbooks adhered to the NCTM 


standards and instructional design criteria. They analyzed lessons on addition and subtraction 


on five textbooks at the third grade level that break “each lesson in six parts, including 
preview, warm-up/introduction, teach, practice, assess, and review” for evaluating the 
standards (p. 322). They found great discrepancy across textbooks in meeting the standards. 

We reviewed several journal articles that evaluated changes in teaching practices 
based on recent curricular reforms in countries beyond the United States. Ampadu (2014) 
examined the impacts of a mathematics curriculum introduced in Ghana in 2007. This 
reform aimed to shift from a teacher-centered approach to more student-centered style that 
emphasized participatory teaching and learning. Using a semi-structured questionnaire for 
teachers and class observations, the study investigated teacher beliefs about their teaching, 
differences between these beliefs and curriculum requirements, and the relationship between 
beliefs and teaching practices. The paper found that “the movement towards a more 
constructivist approach as outlined in the curriculum was not fully evident in most of the 
classrooms observed” (p. 79). 

Bobis and Anderson (2006) evaluated the implementation of reform-oriented 
approaches promoted through the Working Mathematically curriculum for primary school 
children in New South Wales, Australia. Using teacher surveys (based on Ross, et al., 2003), 
they examined which reform-oriented teaching practices primary school teachers reported 
using and the particular teaching practices used for each of the processes of Working 
Mathematically. The study also identified the knowledge and beliefs that were associated 
with teachers who successfully implement Working Mathematically. Following a similar line 
of research, Cavanagh (2006) evaluated the implementation of the new reform-oriented 


syllabus in secondary schools in New South Wales. 


Huntley (2009) described different methods of measuring curriculum implementation. 
The study used the Concerned-Based Adoption Model, “premised on the notion that 
implementation of an innovation (such as a new textbook) can take on many different 
operational forms” (p. 357). An example of a tool is teachers using a textbook's table of 


contents to identify chapters, units, or topics they plan to teach during the academic year. 
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